Immersive teleconferencing and telepresence

ABSTRACT

Embodiments may relate to a user equipment (UE) that is configured to determine a real-time transport protocol (RTP) media flow that includes visual data related to a plurality of images concurrently taken of a location and a supplemental information enhancement (SEI) message that is to be used to display at least a portion of the visual data. The UE is further configured to visually display, based on the visual data and the SEI message, the portion of the visual data to a user of the user device. Other embodiments may be described or claimed.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/439,322, filed Sep. 14, 2021, which is a U.S. National Phaseapplication under 35 U.S.C. § 371 of International Patent ApplicationNo. PCT/US2020/039512, filed on Jun. 25, 2020, which claims the benefitof the priority of U.S. Provisional Patent Application No. 62/866,488,filed on Jun. 25, 2019, the disclosures of each of which areincorporated herein by reference in their entirety.

BACKGROUND

Telepresence and teleconference is becoming more ubiquitous as cellularbandwidths increase. In some use cases, multiple users may join ameeting from a remote location via a user equipment (UE) such as acellular phone, a tablet, a headset, or some other device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a simplified architecture related to a first example usecase of various embodiments herein.

FIG. 2 depicts a simplified alternative architecture related to a secondexample use case of various embodiments herein.

FIG. 3 depicts an overview of a possible receiver architecture, inaccordance with various embodiments herein.

FIG. 4 depicts an example process flow related to embodiments herein.

FIG. 5 illustrates an example architecture of a system of a network, inaccordance with various embodiments.

FIG. 6 illustrates an example of a platform (or “device”) in accordancewith various embodiments.

FIG. 7 illustrates example components of baseband circuitry and radiofront-end modules (RFEM) in accordance with various embodiments.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings which form a part hereof, wherein like numeralsdesignate like parts throughout, and in which is shown by way ofillustration embodiments in which the subject matter of the presentdisclosure may be practiced. It is to be understood that otherembodiments may be utilized and structural or logical changes may bemade without departing from the scope of the present disclosure.Therefore, the following detailed description is not to be taken in alimiting sense.

For the purposes of the present disclosure, the phrase “A or B” means(A), (B), or (A and B). For the purposes of the present disclosure, thephrase “A, B, or C” means (A), (B), (C), (A and B), (A and C), (B andC), or (A, B and C). The description may use the phrases “in anembodiment,” or “in embodiments,” which may each refer to one or more ofthe same or different embodiments. Furthermore, the terms “comprising,”“including,” “having,” and the like, as used with respect to embodimentsof the present disclosure, are synonymous.

The term “coupled with,” along with its derivatives, may be used herein.“Coupled” may mean one or more of the following. “Coupled” may mean thattwo or more elements are in direct physical or electrical contact.However, “coupled” may also mean that two or more elements indirectlycontact each other, but yet still cooperate or interact with each other,and may mean that one or more other elements are coupled or connectedbetween the elements that are said to be coupled with each other. Theterm “directly coupled” may mean that two or elements are in directcontact.

Various operations may be described as multiple discrete operations inturn, in a manner that is most helpful in understanding the claimedsubject matter. However, the order of description should not beconstrued as to imply that these operations are necessarily orderdependent.

Generally, embodiments herein relate to SDP-based and Real-timeTransport Protocol (RTP)-based procedures to support immersiveteleconferencing and telepresence. For example, embodiments may relateto SDP procedures to negotiate immersive media exchange capabilities.Embodiments herein may provide a number of benefits such assimplification of interoperability and SDP handling

FIG. 1 depicts a simplified architecture 100 related to a first exampleuse case of embodiments herein. It will be noted that this use case isdescribed with respect to a conference call with two remote individuals105 a and 105 b, however in other embodiments the architecture mayinclude more or fewer individuals. Specifically, in FIG. 1 , a group ofcolleagues may be having a meeting in conference room 110. The room mayinclude a conference table (for physically present participants), acamera, and a view screen. For the purposes of this disclosure, thecamera will be described as being a “360-degree camera” which is able tocapture video in a full 360 degree view field around the camera.Specifically, the camera may include a plurality of individual camerasor lenses which are able to capture video at different angles or fieldsof view relative to the camera. However, it will be recognized that inother embodiments the camera may not be a full 360-degree camera, butrather may be configured to capture a smaller field of view than a full360-degree field of view.

Two other individuals at 105 a and 105 b may not be in the same locationas the conference room 110, and may wish to join the meeting through aconference call. For example, participants in the conference room 110may use the screen to display a shared presentation and/or video streamscoming from individuals 105 a and 105 b. In this example use case,individual 105 a may join the video conference from a remote locationsuch as their home using a UE with a Head Mounted Display (HMD) and acamera that captures their video. Individual 105 a may enjoy a360-degree view of the conference room 110 by, for example, turningtheir head. Individual 105 b may join the conference from a remotelocation such as an airport using a UE such as a mobile phone.Individual 105 b may also enjoy a 360-degree view of the conference roomon the screen of their mobile phone by turning the phone, and mayfurther use their mobile camera for capturing his own video.

In this use case, individuals 105 a and 105 b may be able to see thescreen in the conference room 110 as part of the 360-degree video. Theymay also have the option to bring into focus one or more of the variousincoming video streams (e.g., presentation or the other remoteparticipant's camera feed) using their own display devices.

Generally, the above-described use case can be realized in two possibleconfigurations. The first configuration is as described above withrespect to FIG. 1 . In this configuration, communication between theconference room 110 and the various individuals 105 a and 105 b (orother individuals which may join using one or more UEs) may be set upwithout the support of media-aware network elements such as a conferenceserver. In this embodiment, communication such as audio or video may betransmitted directly between the conference room 110 and the individuals105 a/105 b by way of a conversational audio/video stream 125.Specifically, the conversational audio or video stream may provide audioor visual data (for example, of an individual 105 a or 105 b) betweenthe conference room and the UE of the individuals 105 a/105 b.

Additionally, the individuals 105 a/105 b may provide viewport-relatedinformation 115 to the conference room 110, which in turn may provide aviewport-dependent audio or video stream at 120. As used herein, theterm “viewport” may relate to a field of view of the conference room 110as displayed by a UE. For example, the term “viewport” may relate to azoom-level, a specific orientation, etc. of the conference room 110.More specifically, an individual such as individuals 105 a/105 b maychange their viewport through a variety of mechanism such as interactionwith the UE (e.g., swiping on touch screen of the UE or some otherinteraction), rotation of the UE, a command such as a verbal command, agesture, etc. This change may be communicated via the viewport-relatedinformation 115. In response, the viewport-dependent audio or videostream 120 may be altered to include a corresponding change to the videoinformation provided to the UEs of individuals 105 a/105 b such as thefield of view, the zoom level, stereo-aspects of the audio stream, etc.

FIG. 2 depicts a simplified alternative architecture 200 related to asecond example use case of various embodiments herein Similarly toarchitecture 100, the architecture 200 may include a conference room 210and individuals 205 a/205 b, which may be similar to conference room 110and individuals 105 a and 105 b. In the second scenario, the call issetup using a network function, which may be performed by either a MediaResource Function (MRF) or a Media Control Unit (MCU). Specifically, theMRF/MCU may be provided by a conference server 230 which may becommunicatively located between the individuals 205 a/205 b and theconference room 210. In some embodiments, the conference server 230 maybe an internet protocol (IP) multimedia subsystem (IMS) server that isoperable to provide MRF or MCU functionality.

In this example use case, the server 230 may receive aviewport-independent stream 240 from the conference room 210.Specifically, the server 230 may receive audio or visual data that isnot based on a specific viewport orientation. The server 230 may also becommunicatively coupled with the conference room 210 to provide aconversational audio or visual data stream 235 which may be generallysimilar to the streams 125 described above with respect to FIG. 1 .

The server 230 may further be communicatively coupled with individuals205 a and 205 b, via data streams such as a viewport-dependent stream220, viewport-related information 215, and a conversational audio/videostream 225 which may be respectively similar to viewport-dependentstream 120, viewport-related information 115, and conversationalaudio/video stream 125.

Typically, the use case of architectures 100 or 200 may enable animmersive experience for individuals 205 a/205 b joiningteleconferencing and telepresence sessions, with two-way audio andone-way immersive video, e.g., a remote user wearing an HMDparticipating in a conference may send audio and optionally 2D video(e.g., of a presentation, screen sharing and/or a capture of the useritself), but receives stereo or immersive voice/audio and immersivevideo captured by an omnidirectional camera in a conference roomconnected to a fixed network.

It will be understood that these architectures are examplearchitectures, and other embodiments may include aspects of botharchitectures, or additional aspects. For example, in some embodimentsan individual's UE may be directly coupled with the conference roomwhile another individual's UE may be communicatively coupled with aserver. In some embodiments, an individual's UE may be communicativelycoupled with a server for viewport-related information, while theindividual's UE may be coupled directly with the conference room for aconversational audio or visual data stream. It will also be understoodthat, as used herein, the concept of “coupled with the conference room”is used to describe a communicative coupling with, for example, a serveror other electronic device that is providing or receiving audio orvisual data from a speaker or camera of the conference room.

The architectures 100 or 200 may have a number of example features. Onesuch feature may be that multiple single-user participants such asindividuals 105 a/105 b/205 a/205 b may be allowed. Communicationsbetween the single users may be structured as multimedia telephonyservice for IMS (MTSI) or Telepresence communications as defined bythird generation partnership project (3GPP) specifications. In otherembodiments, the communications may be structured as multi-stream MTSI(MSMTSI). In embodiments where MSMTSI is used, then media data may betransmitted in separate media streams.

Another such feature may be the presence of a single 360-degree cameraper location in multi-party conference scenarios (e.g., in respectiveones of the conference rooms 110 or 210) involving multiple physicallocations. As noted, the camera may take a number of two-dimensionalimages. The images may then be “stitched” together, or combined, into animage with a broader (e.g., 360-degree) field of view. Variousembodiments may use in-camera stitching, wherein the images are stitchedtogether by the camera itself, or network-based stitching, wherein theimages are stitched together by a server such as server 230.

In the case of camera stitching, stitched immersive video may sent fromthe conference room 210 to the server, for example inviewport-independent stream 240, and then from the server 230 to theindividuals 205 a or 205 b, e.g. through the viewport-dependent stream220, the conversational audio/video stream 225, or both. If the use caseis a one-to-one conversational session between the conference room 210and the user (e.g., individuals 105 a/105 b/205 a/205 b), a server suchas server 230 which may act as a media gateway may not be necessary.

In the use case of network-based stitching, the various two-dimensionalimages may be sent from the conference room 210 to the server 230, forexample in viewport-independent stream 240. The server may performdecoding, stitching, and re-encoding of the images to produce theimmersive video (e.g., the video or image with the larger field ofview), which is then distributed to the remote participants as describedabove.

In some embodiments, it may be desirable for various elements of thearchitectures 100 or 200 such as the camera or electronics in theconference rooms 110/210, UEs of the individuals 105 a/105 b/205 a/205b, or server 230 to support MTSI or IMS telepresence codec, protocol, ortransport capabilities relevant for encoding, delivery, and consumptionof immersive speech/audio and immersive video. It may also be desirablefor elements such as the camera or electronics of the conference rooms110 or 210, or the server 230 to be able to send viewport-dependentstreams, viewport-independent streams, or both. It may also be desirableto structure the architecture such that changes in viewport orientationare delivered, and the relevant viewport-dependent streams are updated,in such a manner as to reduce or eliminate latency-prone signaling, suchas SIP renegotiations. It may also be desirable for the architecture toestablish a suitable coordinate system to be used as the standard way ofcommunicating the orientation of the viewport between various elementsof the architectures 100 or 200.

Generally, embodiments herein may include aspects or elements similar tothe MTSI service architecture depicted in FIG. 4.1 of the 3GPP technicalspecification (TS) 26.114 v16.2.0 (June, 2019) for immersiveteleconferencing. Further, the following may be observed.

For in-camera stitching, stitched immersive video may be sent from theconferencing room to the conferencing server (e.g., MSMTSI MRF) ordirectly to the remote participant (e.g., one-to-one conversation) inone or more RTP streams (e.g., established via SDP). Multiple RTPstreams may be used in case tile or sub-picture based deliveryoptimization is in use. In these embodiments, the RTP streams may be,for example, the viewport-dependent streams 120/220, or theviewport-independent stream 240 of FIG. 1 or 2 .

For network-based stitching, multiple RTP streams may be established(e.g., via SDP, using MSMTSI) between the conferencing server andconference room, e.g. as multiple viewport-independent streams 240, eachof which may carry a particular two-dimensional image or video capture.These RTP streams may then be sent from the conference room to theconferencing server and the conferencing server may perform decoding,stitching, and re-encoding to produce one or more RTP streams containingthe immersive video, which are then distributed to the remoteparticipants (e.g., again via MSMTSI), for example in theviewport-dependent streams 220. Multiple RTP streams may be used for theimmersive video in case tile or sub-picture based delivery optimizationis in use.

FIG. 3 depicts an overview of a possible receiver architecture, inaccordance with various embodiments herein. Specifically, FIG. 3provides an overview of a possible receiver architecture thatreconstructs the spherical video in an MTSI or IMS Telepresence UE suchas may be used by individuals 105 a/105 b/205 a/205 b/etc. It may beunderstood that FIG. 3 may not represent an actual implementation, butrather may be considered to depict a logical set of receiver functions.Generally, some or all of the elements of FIG. 3 may be implemented in abaseband receiver. In other embodiments, various of the elements may beimplemented in or by a processor or element of a radio frequency (RF)transceiver, or elsewhere within the UE.

Initially, an RTP receiver 300 may receive one or more RTP streams 301.The RTP streams 301 may be as received from, for example, a server suchas server 230 or a conference room such as conference rooms 110 or 210as described above. More specifically, the one or more RTP streams 301may be received in a viewport-dependent stream such asviewport-dependent streams 120 or 220. Based on the one or more receivedRTP streams 301, the RTP receiver 300 may parse, possibly decrypt, orother process the one or more RTP streams 301 to generate an elementarystream 303 which is provided to a high-efficiency video coding (HEVC)decoder 305. The HEVC decoder 305 may obtains the decoder output signalfrom the elementary stream 303. The decoder output signal may includeimage data which may be referred to herein as the “texture,” 307. TheHEVC decoder 305 may further obtain decoder metadata 309. The decodermetadata 309 may include one or more supplemental informationenhancement (SEI) messages, e.g., information carried in theomnidirectional video specific SEI messages, to be used by the UE in therendering phase. In particular, the decoder metadata 309 may be used bya texture-to-sphere mapping function 310 to generate a spherical video311 (or part thereof) based on the decoded output signal, e.g., thetexture 307. The viewport may then be generated by a viewport renderingmodule 315 from the spherical video signal 311 (or part thereof) bytaking into account the viewport position information from sensors,display characteristics as well as possibly other metadata such asinitial viewport information. The rendered signal may then be providedto a display of the UE such as a touchscreen, an eyepiece, or some otherdisplay such that the user of the UE may view the rendered image.

For 360-degree video, the potential solutions can consider one or moreof the following principles. Specifically, the RTP stream 301 maycontain an HEVC bitstream with omnidirectional video specific SEImessages. The omnidirectional video specific SEI messages may be similarto those defined in the International Organization forStandardization/International Electrotechnical Commission (ISO/IEC)standard 23008-2 (2017). The elementary stream 303 may be based on theOmnidirectional Media Format (OMAF) specification ISO/IEC 23090-2(2019), clause 10.1.2.2.

Generally, the SEI messages in the elementary stream 303 with decoderrendering metadata may include various information. For example, the SEImessage(s) may include region-wise packing information, e.g., carryingregion-wise packing format indication and also any coveragerestrictions. The SEI message(s) may further include projection mappinginformation, indicating the projection format in use, e.g.,Equirectangular projection (ERP) or Cubemap projection (CMP). The SEImessage(s) may further include information related to padding, e.g.,whether there is padding or guard band in the packed picture. The SEImessage(s) may further include information related to a frame packingarrangement, indicating the frame packing format for stereoscopiccontent. The SEI message(s) may further include content pre-rotationinformation, indicating the amount of sphere rotation, if any, appliedto the sphere signal before projection and region-wise packing at theencoder side.

The output signal, e.g., the decoded picture or “texture” 307, may thenbe rendered using the decoder metadata information 309 related to therelevant SEI messages. The decoder metadata 309 may be used whenperforming rendering operations such as region-wise unpacking,projection de-mapping and rotation toward creating spherical content foreach eye of a user (if desired).

In some embodiments, viewport-dependent processing may be supported forboth point-to-point conversational sessions and multi-party conferencingscenarios. The viewport-dependent processing may be achieved by sendingfrom the MTSI receiver (e.g., a UE of an individual such as individuals105 a/105 b/205 a/205 b) RTP control protocol (RTCP) feedback or RTPheader extension messages with the desired viewport information, forexample in the viewport-related information 115 or 215. Thecorresponding viewport-dependent information may then be encoded andsent by the MTSI sender (e.g., at the conference room 110 inviewport-dependent stream 120) or by the server 230 (e.g., inviewport-dependent stream 220).

This process flow may deliver resolutions higher than aviewport-independent approach for the desired viewport. Generally,viewport-dependent processing based on tiling and sub-picture coding maybe based on RTP/RTCP based protocols that are supported by MTSI andIMS-based telepresence.

For achieving video quality which may be viewed as acceptable for aviewport-dependent virtual reality (VR) service, it may be desirable forthe video codecs for VR support in MTSI and IMS telepresence to bealigned with OMAF, the 3GPP TS 26.118 v 15.1.0 (December, 2018), orboth. It may be desirable for both MTSI client (as may be used by theUE) and MTSI gateway (as may be used by a server 230) codec requirementsto be aligned with these recommended video codec requirements for VRsupport.

Procedures related to the negotiation of SEI messages for carriage ofdecoder rendering metadata may be similar to those described in theInternet Engineering Task Force (IETF) Request For Comment (RFC) 7798(March, 2016) on the RTP payload format for HEVC. In particular theprocedures may relate to exposing SEI messages related to decoderrendering metadata for omnidirectional media in the SDP using the‘sprop-sei’ parameter, which may allow for the conveyance of one or moreSEI messages that describe bitstream characteristics. When present, adecoder may rely on the bitstream characteristics that are described inthe SEI messages for the entire duration of the session. In someembodiments, both MTSI clients and MTSI gateways may support RTP payloadformats for VR support.

As noted, SEI messages may be present in the decoder metadata 309. TheSEI messages may include one or more of: the equirectangular projectionSEI message; the cubemap projection SEI message; the sphere rotation SEImessage; and the region-wise packing SEI message. For stereoscopic videosupport, in either one-to-one video telephony scenarios or multi-partyvideo conferencing scenarios, support of a subset of the frame packingarrangement SEI message may be desirable.

FIG. 4 depicts an example process flow related to embodiments herein. Itwill be understood that, for in-camera stitching, stitched immersivevideo may be sent from the conference room (e.g., conference rooms 110or 210) to the server (e.g., server 230) or directly to the remoteparticipant (e.g., to individuals 105 a/105 b/205 a/205 b) in one ormore RTP streams (e.g., established via SDP) as described above.Additionally, in some embodiments, multiple RTP streams may be used incase tile or sub-picture based delivery optimization is in use.

There may be 3 communicative elements present in the process flow ofFIG. 4 . Specifically, the process flow may include a UE of anindividual at 405, which may be similar to, for example, the UE ofindividuals 105 a/105 b/205 a/205 b/etc. The process flow may furtherinclude a server 430, which may be similar to, for example, server 230.The process flow may further include an electronic device at aconference room 410 which may be similar to, for example, conferencerooms 110 or 210.

Initially, at 402, the UE at 405 may send an SDP offer to the server 430indicating immersive media capabilities including 360-degree videosupport. In some embodiments, the UE at 405 may also includeviewport-dependent processing capability in the SDP offer, e.g., basedon various embodiments described herein. Two or more RTP streams may beincluded in the SDP offer at 402 in case viewport-dependent processingis offered, e.g. one RTP stream for the base 360-degree video andanother viewport-optimized RTP stream, with the high quality 360-degreevideo corresponding to the desired viewport.

At 404, the server 430 may respond to the UE 405 with an SDP answerconfirming immersive media capabilities of the server 430, including360-degree video support. In some embodiments, the server 430 may alsoinclude an indication of viewport-dependent processing capability in theSDP answer at 404. In case viewport-dependent processing is accepted orincluded, the SDP answer from the server 430 may include multiple RTPstreams.

At 406, the conference room 410 may provide viewport-independentinformation to the server 430, and then at 408 the server 430 may streamthe RTP media flow with immersive media including 360-degree video tothe UE at 405. 360-degree video transmission could be based on the RTPpayload formats for HEVC that carry SEI messages

In some embodiments, at 411, the UE at 405 may signal desired viewportinformation using a dedicated RTCP feedback message. In this embodiment,at 412, the server 430 may respond to UE 405. Information on theactually transmitted viewport may also be included in the RTP mediaflow. In case two RTP streams are negotiated, then theviewport-optimized RTP stream containing the high quality 360-degreevideo may contain this information.

It will be understood that this embodiment is intended as one exampleembodiment, and other embodiments may include more or fewer elements,elements in a different order than depicted, etc. It will also beunderstood that although elements herein are described as relating to360-degree video, in other embodiments the media flows may be related toa narrower field of view as described above.

FIG. 5 illustrates an example architecture of a system 500 of a network,in accordance with various embodiments. The following description isprovided for an example system 500 that operates in conjunction with theLTE system standards and 5G or NR system standards as provided by 3GPPtechnical specifications. However, the example embodiments are notlimited in this regard and the described embodiments may apply to othernetworks that benefit from the principles described herein, such asfuture 3GPP systems (e.g., Sixth Generation (6G)) systems, IEEE 802.16protocols (e.g., WMAN, WiMAX, etc.), or the like.

As shown by FIG. 5 , the system 500 includes UE 501 a and UE 501 b(collectively referred to as “UEs 501” or “UE 501”), which may besimilar to, for example, the UEs of individuals 105 a/105 b/205 a/205 c,or the UE described at 405. In this example, UEs 501 are illustrated assmartphones (e.g., handheld touchscreen mobile computing devicesconnectable to one or more cellular networks), but may also comprise anymobile or non-mobile computing device, such as consumer electronicsdevices, cellular phones, smartphones, feature phones, tablet computers,wearable computer devices, personal digital assistants (PDAs), pagers,wireless handsets, desktop computers, laptop computers, in-vehicleinfotainment (IVI), in-car entertainment (ICE) devices, an InstrumentCluster, head-up display (HUD) devices, onboard diagnostic (OBD)devices, dashtop mobile equipment (DME), mobile data terminals (MDTs),Electronic Engine Management System (EEMS), electronic/engine controlunits (ECUs), electronic/engine control modules (ECMs), embeddedsystems, microcontrollers, control modules, engine management systems(EMS), networked or “smart” appliances, MTC devices, M2M, IoT devices,and/or the like.

In some embodiments, any of the UEs 501 may be IoT UEs, which maycomprise a network access layer designed for low-power IoT applicationsutilizing short-lived UE connections. An IoT UE can utilize technologiessuch as M2M or MTC for exchanging data with an MTC server or device viaa PLMN, ProSe or D2D communication, sensor networks, or IoT networks.The M2M or MTC exchange of data may be a machine-initiated exchange ofdata. An IoT network describes interconnecting IoT UEs, which mayinclude uniquely identifiable embedded computing devices (within theInternet infrastructure), with short-lived connections. The IoT UEs mayexecute background applications (e.g., keep-alive messages, statusupdates, etc.) to facilitate the connections of the IoT network.

The UEs 501 may be configured to connect, for example, communicativelycouple, with an RAN 510. In embodiments, the RAN 510 may be an NG RAN ora 5G RAN, an E-UTRAN, or a legacy RAN, such as a UTRAN or GERAN. As usedherein, the term “NG RAN” or the like may refer to a RAN 510 thatoperates in an NR or 5G system 500, and the term “E-UTRAN” or the likemay refer to a RAN 510 that operates in an LTE or 4G system 500. The UEs501 utilize connections (or channels) 503 and 504, respectively, each ofwhich comprises a physical communications interface or layer (discussedin further detail below).

In this example, the connections 503 and 504 are illustrated as an airinterface to enable communicative coupling, and can be consistent withcellular communications protocols, such as a GSM protocol, a CDMAnetwork protocol, a PTT protocol, a POC protocol, a UMTS protocol, a3GPP LTE protocol, a 5G protocol, a NR protocol, and/or any of the othercommunications protocols discussed herein. In embodiments, the UEs 501may directly exchange communication data via a ProSe interface 505. TheProSe interface 505 may alternatively be referred to as a SL interface505 and may comprise one or more logical channels, including but notlimited to a PSCCH, a PSSCH, a PSDCH, and a PSBCH.

The UE 501 b is shown to be configured to access an AP 506 (alsoreferred to as “WLAN node 506,” “WLAN 506,” “WLAN Termination 506,” “WT506” or the like) via connection 507. The connection 507 can comprise alocal wireless connection, such as a connection consistent with any IEEE802.11 protocol, wherein the AP 506 would comprise a wireless fidelity(Wi-Fi®) router. In this example, the AP 506 is shown to be connected tothe Internet without connecting to the core network of the wirelesssystem (described in further detail below). In various embodiments, theUE 501 b, RAN 510, and AP 506 may be configured to utilize LWA operationand/or LWIP operation. The LWA operation may involve the UE 501 b inRRC_CONNECTED being configured by a RAN node 511 a-b to utilize radioresources of LTE and WLAN. LWIP operation may involve the UE 501 b usingWLAN radio resources (e.g., connection 507) via IPsec protocol tunnelingto authenticate and encrypt packets (e.g., IP packets) sent over theconnection 507. IPsec tunneling may include encapsulating the entiretyof original IP packets and adding a new packet header, therebyprotecting the original header of the IP packets.

The RAN 510 can include one or more AN nodes or RAN nodes 511 a and 511b (collectively referred to as “RAN nodes 511” or “RAN node 511”) thatenable the connections 503 and 504. As used herein, the terms “accessnode,” “access point,” or the like may describe equipment that providesthe radio baseband functions for data and/or voice connectivity betweena network and one or more users. These access nodes can be referred toas BS, gNBs, RAN nodes, eNBs, NodeBs, RSUs, TRxPs or TRPs, and so forth,and can comprise ground stations (e.g., terrestrial access points) orsatellite stations providing coverage within a geographic area (e.g., acell). As used herein, the term “NG RAN node” or the like may refer to aRAN node 511 that operates in an NR or 5G system 500 (for example, agNB), and the term “E-UTRAN node” or the like may refer to a RAN node511 that operates in an LTE or 4G system 500 (e.g., an eNB). Accordingto various embodiments, the RAN nodes 511 may be implemented as one ormore of a dedicated physical device such as a macrocell base station,and/or a low-power (LP) base station for providing femtocells, picocellsor other like cells having smaller coverage areas, smaller usercapacity, or higher bandwidth compared to macrocells.

In some embodiments, all or parts of the RAN nodes 511 may beimplemented as one or more software entities running on server computersas part of a virtual network, which may be referred to as a CRAN and/ora virtual baseband unit pool (vBBUP). In these embodiments, the CRAN orvBBUP may implement a RAN function split, such as a PDCP split whereinRRC and PDCP layers are operated by the CRAN/vBBUP and other L2 protocolentities are operated by individual RAN nodes 511; a MAC/PHY splitwherein RRC, PDCP, RLC, and MAC layers are operated by the CRAN/vBBUPand the PHY layer is operated by individual RAN nodes 511; or a “lowerPHY” split wherein RRC, PDCP, RLC, MAC layers and upper portions of thePHY layer are operated by the CRAN/vBBUP and lower portions of the PHYlayer are operated by individual RAN nodes 511. This virtualizedframework allows the freed-up processor cores of the RAN nodes 511 toperform other virtualized applications. In some implementations, anindividual RAN node 511 may represent individual gNB-DUs that areconnected to a gNB-CU via individual F1 interfaces (not shown by FIG. 5). In these implementations, the gNB-DUs may include one or more remoteradio heads or RFEMs, and the gNB-CU may be operated by a server that islocated in the RAN 510 (not shown) or by a server pool in a similarmanner as the CRAN/vBBUP. Additionally or alternatively, one or more ofthe RAN nodes 511 may be next generation eNBs (ng-eNBs), which are RANnodes that provide E-UTRA user plane and control plane protocolterminations toward the UEs 501, and are connected to a 5GC via an NGinterface (discussed infra).

In V2X scenarios one or more of the RAN nodes 511 may be or act as RSUs.The term “Road Side Unit” or “RSU” may refer to any transportationinfrastructure entity used for V2X communications. An RSU may beimplemented in or by a suitable RAN node or a stationary (or relativelystationary) UE, where an RSU implemented in or by a UE may be referredto as a “UE-type RSU,” an RSU implemented in or by an eNB may bereferred to as an “eNB-type RSU,” an RSU implemented in or by a gNB maybe referred to as a “gNB-type RSU,” and the like. In one example, an RSUis a computing device coupled with RF circuitry located on a roadsidethat provides connectivity support to passing vehicle UEs 501 (vUEs501). The RSU may also include internal data storage circuitry to storeintersection map geometry, traffic statistics, media, as well asapplications/software to sense and control ongoing vehicular andpedestrian traffic. The RSU may operate on the 5.9 GHz DirectShort-Range Communications (DSRC) band to provide very low latencycommunications required for high-speed events, such as crash avoidance,traffic warnings, and the like. Additionally or alternatively, the RSUmay operate on the cellular V2X band to provide the aforementioned lowlatency communications, as well as other cellular communicationsservices. Additionally or alternatively, the RSU may operate as a Wi-Fihotspot (2.4 GHz band) and/or provide connectivity to one or morecellular networks to provide uplink and downlink communications. Thecomputing device(s) and some or all of the radiofrequency circuitry ofthe RSU may be packaged in a weatherproof enclosure suitable for outdoorinstallation, and may include a network interface controller to providea wired connection (e.g., Ethernet) to a traffic signal controllerand/or a backhaul network.

Any of the RAN nodes 511 can terminate the air interface protocol andcan be the first point of contact for the UEs 501. In some embodiments,any of the RAN nodes 511 can fulfill various logical functions for theRAN 510 including, but not limited to, radio network controller (RNC)functions such as radio bearer management, uplink and downlink dynamicradio resource management and data packet scheduling, and mobilitymanagement.

In embodiments, the UEs 501 can be configured to communicate using OFDMcommunication signals with each other or with any of the RAN nodes 511over a multicarrier communication channel in accordance with variouscommunication techniques, such as, but not limited to, an OFDMAcommunication technique (e.g., for downlink communications) or a SC-FDMAcommunication technique (e.g., for uplink and ProSe or sidelinkcommunications), although the scope of the embodiments is not limited inthis respect. The OFDM signals can comprise a plurality of orthogonalsubcarriers.

In some embodiments, a downlink resource grid can be used for downlinktransmissions from any of the RAN nodes 511 to the UEs 501, while uplinktransmissions can utilize similar techniques. The grid can be atime-frequency grid, called a resource grid or time-frequency resourcegrid, which is the physical resource in the downlink in each slot. Sucha time-frequency plane representation is a common practice for OFDMsystems, which makes it intuitive for radio resource allocation. Eachcolumn and each row of the resource grid corresponds to one OFDM symboland one OFDM subcarrier, respectively. The duration of the resource gridin the time domain corresponds to one slot in a radio frame. Thesmallest time-frequency unit in a resource grid is denoted as a resourceelement. Each resource grid comprises a number of resource blocks, whichdescribe the mapping of certain physical channels to resource elements.Each resource block comprises a collection of resource elements; in thefrequency domain, this may represent the smallest quantity of resourcesthat currently can be allocated. There are several different physicaldownlink channels that are conveyed using such resource blocks.

According to various embodiments, the UEs 501 and the RAN nodes 511communicate data (for example, transmit and receive) data over alicensed medium (also referred to as the “licensed spectrum” and/or the“licensed band”) and an unlicensed shared medium (also referred to asthe “unlicensed spectrum” and/or the “unlicensed band”). The licensedspectrum may include channels that operate in the frequency range ofapproximately 400 MHz to approximately 3.8 GHz, whereas the unlicensedspectrum may include the 5 GHz band.

To operate in the unlicensed spectrum, the UEs 501 and the RAN nodes 511may operate using LAA, eLAA, and/or feLAA mechanisms. In theseimplementations, the UEs 501 and the RAN nodes 511 may perform one ormore known medium-sensing operations and/or carrier-sensing operationsin order to determine whether one or more channels in the unlicensedspectrum is unavailable or otherwise occupied prior to transmitting inthe unlicensed spectrum. The medium/carrier-sensing operations may beperformed according to a listen-before-talk (LBT) protocol.

LBT is a mechanism whereby equipment (for example, UEs 501 RAN nodes511, etc.) senses a medium (for example, a channel or carrier frequency)and transmits when the medium is sensed to be idle (or when a specificchannel in the medium is sensed to be unoccupied). The medium-sensingoperation may include CCA, which utilizes at least ED to determine thepresence or absence of other signals on a channel in order to determineif a channel is occupied or clear. This LBT mechanism allowscellular/LAA networks to coexist with incumbent systems in theunlicensed spectrum and with other LAA networks. ED may include sensingRF energy across an intended transmission band for a period of time andcomparing the sensed RF energy to a predefined or configured threshold.

Typically, the incumbent systems in the 5 GHz band are WLANs based onIEEE 802.11 technologies. WLAN employs a contention-based channel accessmechanism, called CSMA/CA. Here, when a WLAN node (e.g., a mobilestation (MS) such as UE 501, AP 506, or the like) intends to transmit,the WLAN node may first perform CCA before transmission. Additionally, abackoff mechanism is used to avoid collisions in situations where morethan one WLAN node senses the channel as idle and transmits at the sametime. The backoff mechanism may be a counter that is drawn randomlywithin the CWS, which is increased exponentially upon the occurrence ofcollision and reset to a minimum value when the transmission succeeds.The LBT mechanism designed for LAA is somewhat similar to the CSMA/CA ofWLAN. In some implementations, the LBT procedure for DL or ULtransmission bursts including PDSCH or PUSCH transmissions,respectively, may have an LAA contention window that is variable inlength between X and Y ECCA slots, where X and Y are minimum and maximumvalues for the CWSs for LAA. In one example, the minimum CWS for an LAAtransmission may be 9 microseconds (μs); however, the size of the CWSand a MCOT (for example, a transmission burst) may be based ongovernmental regulatory requirements.

The LAA mechanisms are built upon CA technologies of LTE-Advancedsystems. In CA, each aggregated carrier is referred to as a CC. A CC mayhave a bandwidth of 1.4, 3, 5, 10, 15 or 20 MHz and a maximum of fiveCCs can be aggregated, and therefore, a maximum aggregated bandwidth is100 MHz. In FDD systems, the number of aggregated carriers can bedifferent for DL and UL, where the number of UL CCs is equal to or lowerthan the number of DL component carriers. In some cases, individual CCscan have a different bandwidth than other CCs. In TDD systems, thenumber of CCs as well as the bandwidths of each CC is usually the samefor DL and UL.

CA also comprises individual serving cells to provide individual CCs.The coverage of the serving cells may differ, for example, because CCson different frequency bands will experience different pathloss. Aprimary service cell or PCell may provide a PCC for both UL and DL, andmay handle RRC and NAS related activities. The other serving cells arereferred to as SCells, and each SCell may provide an individual SCC forboth UL and DL. The SCCs may be added and removed as required, whilechanging the PCC may require the UE 501 to undergo a handover. In LAA,eLAA, and feLAA, some or all of the SCells may operate in the unlicensedspectrum (referred to as “LAA SCells”), and the LAA SCells are assistedby a PCell operating in the licensed spectrum. When a UE is configuredwith more than one LAA SCell, the UE may receive UL grants on theconfigured LAA SCells indicating different PUSCH starting positionswithin a same subframe.

The PDSCH carries user data and higher-layer signaling to the UEs 501.The PDCCH carries information about the transport format and resourceallocations related to the PDSCH channel, among other things. It mayalso inform the UEs 501 about the transport format, resource allocation,and HARQ information related to the uplink shared channel Typically,downlink scheduling (assigning control and shared channel resourceblocks to the UE 501 b within a cell) may be performed at any of the RANnodes 511 based on channel quality information fed back from any of theUEs 501. The downlink resource assignment information may be sent on thePDCCH used for (e.g., assigned to) each of the UEs 501.

The PDCCH uses CCEs to convey the control information. Before beingmapped to resource elements, the PDCCH complex-valued symbols may firstbe organized into quadruplets, which may then be permuted using asub-block interleaver for rate matching. Each PDCCH may be transmittedusing one or more of these CCEs, where each CCE may correspond to ninesets of four physical resource elements known as REGs. Four QuadraturePhase Shift Keying (QPSK) symbols may be mapped to each REG. The PDCCHcan be transmitted using one or more CCEs, depending on the size of theDCI and the channel condition. There can be four or more different PDCCHformats defined in LTE with different numbers of CCEs (e.g., aggregationlevel, L=1, 2, 4, or 8).

Some embodiments may use concepts for resource allocation for controlchannel information that are an extension of the above-describedconcepts. For example, some embodiments may utilize an EPDCCH that usesPDSCH resources for control information transmission. The EPDCCH may betransmitted using one or more ECCEs Similar to above, each ECCE maycorrespond to nine sets of four physical resource elements known as anEREG. An ECCE may have other numbers of EREGs in some situations.

The RAN nodes 511 may be configured to communicate with one another viainterface 512. In embodiments where the system 500 is an LTE system(e.g., when CN 520 is an EPC), the interface 512 may be an X2 interface512. The X2 interface may be defined between two or more RAN nodes 511(e.g., two or more eNBs and the like) that connect to EPC 520, and/orbetween two eNBs connecting to EPC 520. In some implementations, the X2interface may include an X2 user plane interface (X2-U) and an X2control plane interface (X2-C). The X2-U may provide flow controlmechanisms for user data packets transferred over the X2 interface, andmay be used to communicate information about the delivery of user databetween eNBs. For example, the X2-U may provide specific sequence numberinformation for user data transferred from a MeNB to an SeNB;information about successful in sequence delivery of PDCP PDUs to a UE501 from an SeNB for user data; information of PDCP PDUs that were notdelivered to a UE 501; information about a current minimum desiredbuffer size at the SeNB for transmitting to the UE user data; and thelike. The X2-C may provide intra-LTE access mobility functionality,including context transfers from source to target eNBs, user planetransport control, etc.; load management functionality; as well asinter-cell interference coordination functionality.

In embodiments where the system 500 is a 5G or NR system, the interface512 may be an Xn interface 512. The Xn interface is defined between twoor more RAN nodes 511 (e.g., two or more gNBs and the like) that connectto 5GC 520, between a RAN node 511 (e.g., a gNB) connecting to 5GC 520and an eNB, and/or between two eNBs connecting to 5GC 520. In someimplementations, the Xn interface may include an Xn user plane (Xn-U)interface and an Xn control plane (Xn-C) interface. The Xn-U may providenon-guaranteed delivery of user plane PDUs and support/provide dataforwarding and flow control functionality. The Xn-C may providemanagement and error handling functionality, functionality to manage theXn-C interface; mobility support for UE 501 in a connected mode (e.g.,CM-CONNECTED) including functionality to manage the UE mobility forconnected mode between one or more RAN nodes 511. The mobility supportmay include context transfer from an old (source) serving RAN node 511to new (target) serving RAN node 511; and control of user plane tunnelsbetween old (source) serving RAN node 511 to new (target) serving RANnode 511. A protocol stack of the Xn-U may include a transport networklayer built on IP transport layer, and a GTP-U layer on top of a UDPand/or IP layer(s) to carry user plane PDUs. The Xn-C protocol stack mayinclude an application layer signaling protocol (referred to as XnApplication Protocol (Xn-AP)) and a transport network layer that isbuilt on SCTP. The SCTP may be on top of an IP layer, and may providethe guaranteed delivery of application layer messages. In the transportIP layer, point-to-point transmission is used to deliver the signalingPDUs. In other implementations, the Xn-U protocol stack and/or the Xn-Cprotocol stack may be same or similar to the user plane and/or controlplane protocol stack(s) shown and described herein.

The RAN 510 is shown to be communicatively coupled to a core network—inthis embodiment, core network (CN) 520. The CN 520 may comprise aplurality of network elements 522, which are configured to offer variousdata and telecommunications services to customers/subscribers (e.g.,users of UEs 501) who are connected to the CN 520 via the RAN 510. Thecomponents of the CN 520 may be implemented in one physical node orseparate physical nodes including components to read and executeinstructions from a machine-readable or computer-readable medium (e.g.,a non-transitory machine-readable storage medium). In some embodiments,NFV may be utilized to virtualize any or all of the above-describednetwork node functions via executable instructions stored in one or morecomputer-readable storage mediums (described in further detail below). Alogical instantiation of the CN 520 may be referred to as a networkslice, and a logical instantiation of a portion of the CN 520 may bereferred to as a network sub-slice. NFV architectures andinfrastructures may be used to virtualize one or more network functions,alternatively performed by proprietary hardware, onto physical resourcescomprising a combination of industry-standard server hardware, storagehardware, or switches. In other words, NFV systems can be used toexecute virtual or reconfigurable implementations of one or more EPCcomponents/functions.

Generally, the application server 530 may be an element offeringapplications that use IP bearer resources with the core network (e.g.,UMTS PS domain, LTE PS data services, etc.). The application server 530can also be configured to support one or more communication services(e.g., VoIP sessions, PTT sessions, group communication sessions, socialnetworking services, etc.) for the UEs 501 via the EPC 520.

In embodiments, the CN 520 may be a 5GC (referred to as “5GC 520” or thelike), and the RAN 510 may be connected with the CN 520 via an NGinterface 513. In embodiments, the NG interface 513 may be split intotwo parts, an NG user plane (NG-U) interface 514, which carries trafficdata between the RAN nodes 511 and a UPF, and the S1 control plane(NG-C) interface 515, which is a signaling interface between the RANnodes 511 and AMFs.

In embodiments, the CN 520 may be a 5G CN (referred to as “5GC 520” orthe like), while in other embodiments, the CN 520 may be an EPC). WhereCN 520 is an EPC (referred to as “EPC 520” or the like), the RAN 510 maybe connected with the CN 520 via an S1 interface 513. In embodiments,the S1 interface 513 may be split into two parts, an S1 user plane(S1-U) interface 514, which carries traffic data between the RAN nodes511 and the S-GW, and the S1-MME interface 515, which is a signalinginterface between the RAN nodes 511 and MMEs.

FIG. 6 illustrates an example of a platform 600 (or “device 600”) inaccordance with various embodiments. In embodiments, the computerplatform 600 may be suitable for use as UEs 501, application servers530, the UEs of individuals 105 a/105 b/205 a/205 b or at 405, or anyother element/device discussed herein. The platform 600 may include anycombinations of the components shown in the example. The components ofplatform 600 may be implemented as integrated circuits (ICs), portionsthereof, discrete electronic devices, or other modules, logic, hardware,software, firmware, or a combination thereof adapted in the computerplatform 600, or as components otherwise incorporated within a chassisof a larger system. The block diagram of FIG. 6 is intended to show ahigh level view of components of the computer platform 600. However,some of the components shown may be omitted, additional components maybe present, and different arrangement of the components shown may occurin other implementations.

Application circuitry 605 includes circuitry such as, but not limited toone or more processors (or processor cores), cache memory, and one ormore of LDOs, interrupt controllers, serial interfaces such as SPI, I2Cor universal programmable serial interface module, RTC, timer-countersincluding interval and watchdog timers, general purpose I/O, memory cardcontrollers such as Secure Digital (SD) MMC or similar, USB interfaces,MIPI interfaces, and JTAG test access ports. The processors (or cores)of the application circuitry 605 may be coupled with or may includememory/storage elements and may be configured to execute instructionsstored in the memory/storage to enable various applications or operatingsystems to run on the system 600. In some implementations, thememory/storage elements may be on-chip memory circuitry, which mayinclude any suitable volatile and/or non-volatile memory, such as DRAM,static random access memory (SRAM), erasable programmable read-onlymemory (EPROM), electrically erasable programmable read-only memory(EEPROM), flash memory, solid-state memory, and/or any other type ofmemory device technology, such as those discussed herein.

As examples, the processor(s) of application circuitry 605 may includean Intel® Architecture Core™ based processor, such as a Quark™, anAtom™, an i3, an i5, an i7, or an MCU-class processor, or another suchprocessor available from Intel® Corporation, Santa Clara, Calif. Theprocessors of the application circuitry 605 may also be one or more ofAdvanced Micro Devices (AMD) Ryzen® processor(s) or AcceleratedProcessing Units (APUs); A5-A9 processor(s) from Apple® Inc.,Snapdragon™ processor(s) from Qualcomm® Technologies, Inc., TexasInstruments, Inc.® Open Multimedia Applications Platform (OMAP)™processor(s); a MIPS-based design from MIPS Technologies, Inc. such asMIPS Warrior M-class, Warrior I-class, and Warrior P-class processors;an ARM-based design licensed from ARM Holdings, Ltd., such as the ARMCortex-A, Cortex-R, and Cortex-M family of processors; or the like. Insome implementations, the application circuitry 605 may be a part of asystem on a chip (SoC) in which the application circuitry 605 and othercomponents are formed into a single integrated circuit, or a singlepackage, such as the Edison™ or Galileo™ SoC boards from Intel®Corporation.

Additionally or alternatively, application circuitry 605 may includecircuitry such as, but not limited to, one or more a field-programmabledevices (FPDs) such as FPGAs and the like; programmable logic devices(PLDs) such as complex PLDs (CPLDs), high-capacity PLDs (HCPLDs), andthe like; ASICs such as structured ASICs and the like; programmable SoCs(PSoCs); and the like. In such embodiments, the circuitry of applicationcircuitry 605 may comprise logic blocks or logic fabric, and otherinterconnected resources that may be programmed to perform variousfunctions, such as the procedures, methods, functions, etc. of thevarious embodiments discussed herein. In such embodiments, the circuitryof application circuitry 605 may include memory cells (e.g., EPROM,EEPROM, flash memory, static memory (e.g., SRAM, anti-fuses, etc.)) usedto store logic blocks, logic fabric, data, etc. in look-up tables (LUTs)and the like.

The baseband circuitry 610 may be implemented, for example, as asolder-down substrate including one or more integrated circuits, asingle packaged integrated circuit soldered to a main circuit board or amulti-chip module containing two or more integrated circuits. Thevarious hardware electronic elements of baseband circuitry 610 arediscussed infra with regard to FIG. 7 .

The RFEMs 615 may comprise a millimeter wave (mmWave) RFEM and one ormore sub-mmWave radio frequency integrated circuits (RFICs). In someimplementations, the one or more sub-mmWave RFICs may be physicallyseparated from the mmWave RFEM. The RFICs may include connections to oneor more antennas or antenna arrays (see e.g., antenna array 711 of FIG.7 infra), and the RFEM may be connected to multiple antennas. Inalternative implementations, both mmWave and sub-mmWave radio functionsmay be implemented in the same physical RFEM 615, which incorporatesboth mmWave antennas and sub-mmWave.

The memory circuitry 620 may include any number and type of memorydevices used to provide for a given amount of system memory. Asexamples, the memory circuitry 620 may include one or more of volatilememory including random access memory (RAM), dynamic RAM (DRAM) and/orsynchronous dynamic RAM (SDRAM), and non-volatile memory (NVM) includinghigh-speed electrically erasable memory (commonly referred to as flashmemory), phase change random access memory (PRAM), magnetoresistiverandom access memory (MRAM), etc. The memory circuitry 620 may bedeveloped in accordance with a Joint Electron Devices EngineeringCouncil (JEDEC) low-power double data rate (LPDDR)-based design, such asLPDDR2, LPDDR3, LPDDR4, or the like. Memory circuitry 620 may beimplemented as one or more of solder-down packaged integrated circuits,single die package, dual die package (DDP) or quad die package (Q17P),socketed memory modules, dual inline memory modules (DIMMs) includingmicroDIMMs or MiniDIMMs, and/or soldered onto a motherboard via a ballgrid array (BGA). In LP implementations, the memory circuitry 620 may beon-die memory or registers associated with the application circuitry605. To provide for persistent storage of information such as data,applications, operating systems and so forth, memory circuitry 620 mayinclude one or more mass storage devices, which may include, inter alia,a solid state disk drive (SSDD), hard disk drive (HDD), a micro HDD,resistance change memories, phase change memories, holographic memories,or chemical memories, among others. For example, the computer platform600 may incorporate the three-dimensional (3D) cross-point (XPOINT)memories from Intel® and Micron®.

Removable memory circuitry 623 may include devices, circuitry,enclosures/housings, ports or receptacles, etc. used to couple portabledata storage devices with the platform 600. These portable data storagedevices may be used for mass storage purposes, and may include, forexample, flash memory cards (e.g., SD cards, microSD cards, xD picturecards, and the like), and USB flash drives, optical discs, externalHDDs, and the like.

The platform 600 may also include interface circuitry (not shown) thatis used to connect external devices with the platform 600. The externaldevices connected to the platform 600 via the interface circuitryinclude sensor circuitry 621 and electro-mechanical components (EMCs)622, as well as removable memory devices coupled to removable memorycircuitry 623.

The sensor circuitry 621 include devices, modules, or subsystems whosepurpose is to detect events or changes in its environment and send theinformation (sensor data) about the detected events to some other adevice, module, subsystem, etc. Examples of such sensors include, interalia, inertia measurement units (IMUs) comprising accelerometers,gyroscopes, and/or magnetometers; microelectromechanical systems (MEMS)or nanoelectromechanical systems (NEMS) comprising 3-axisaccelerometers, 3-axis gyroscopes, and/or magnetometers; level sensors;flow sensors; temperature sensors (e.g., thermistors); pressure sensors;barometric pressure sensors; gravimeters; altimeters; image capturedevices (e.g., cameras or lensless apertures); light detection andranging (LiDAR) sensors; proximity sensors (e.g., infrared radiationdetector and the like), depth sensors, ambient light sensors, ultrasonictransceivers; microphones or other like audio capture devices; etc.

EMCs 622 include devices, modules, or subsystems whose purpose is toenable platform 600 to change its state, position, and/or orientation,or move or control a mechanism or (sub)system. Additionally, EMCs 622may be configured to generate and send messages/signaling to othercomponents of the platform 600 to indicate a current state of the EMCs622. Examples of the EMCs 622 include one or more power switches, relaysincluding electro-mechanical relays (EMRs) and/or solid state relays(SSRs), actuators (e.g., valve actuators, etc.), an audible soundgenerator, a visual warning device, motors (e.g., DC motors, steppermotors, etc.), wheels, thrusters, propellers, claws, clamps, hooks,and/or other like EMCs. In embodiments, platform 600 is configured tooperate one or more EMCs 622 based on one or more captured events and/orinstructions or control signals received from a service provider and/orvarious clients.

In some implementations, the interface circuitry may connect theplatform 600 with positioning circuitry 645. The positioning circuitry645 includes circuitry to receive and decode signalstransmitted/broadcasted by a positioning network of a GNSS. Examples ofnavigation satellite constellations (or GNSS) include United States'GPS, Russia's GLONASS, the European Union's Galileo system, China'sBeiDou Navigation Satellite System, a regional navigation system or GNSSaugmentation system (e.g., NAVIC), Japan's QZSS, France's DORIS, etc.),or the like. The positioning circuitry 645 comprises various hardwareelements (e.g., including hardware devices such as switches, filters,amplifiers, antenna elements, and the like to facilitate OTAcommunications) to communicate with components of a positioning network,such as navigation satellite constellation nodes. In some embodiments,the positioning circuitry 645 may include a Micro-PNT IC that uses amaster timing clock to perform position tracking/estimation without GNSSassistance. The positioning circuitry 645 may also be part of, orinteract with, the baseband circuitry 610 and/or RFEMs 615 tocommunicate with the nodes and components of the positioning network.The positioning circuitry 645 may also provide position data and/or timedata to the application circuitry 605, which may use the data tosynchronize operations with various infrastructure (e.g., radio basestations), for turn-by-turn navigation applications, or the like

In some implementations, the interface circuitry may connect theplatform 600 with Near-Field Communication (NFC) circuitry 640. NFCcircuitry 640 is configured to provide contactless, short-rangecommunications based on radio frequency identification (RFID) standards,wherein magnetic field induction is used to enable communication betweenNFC circuitry 640 and NFC-enabled devices external to the platform 600(e.g., an “NFC touchpoint”). NFC circuitry 640 comprises an NFCcontroller coupled with an antenna element and a processor coupled withthe NFC controller. The NFC controller may be a chip/IC providing NFCfunctionalities to the NFC circuitry 640 by executing NFC controllerfirmware and an NFC stack. The NFC stack may be executed by theprocessor to control the NFC controller, and the NFC controller firmwaremay be executed by the NFC controller to control the antenna element toemit short-range RF signals. The RF signals may power a passive NFC tag(e.g., a microchip embedded in a sticker or wristband) to transmitstored data to the NFC circuitry 640, or initiate data transfer betweenthe NFC circuitry 640 and another active NFC device (e.g., a smartphoneor an NFC-enabled POS terminal) that is proximate to the platform 600.

The driver circuitry 646 may include software and hardware elements thatoperate to control particular devices that are embedded in the platform600, attached to the platform 600, or otherwise communicatively coupledwith the platform 600. The driver circuitry 646 may include individualdrivers allowing other components of the platform 600 to interact withor control various input/output (I/O) devices that may be presentwithin, or connected to, the platform 600. For example, driver circuitry646 may include a display driver to control and allow access to adisplay device, a touchscreen driver to control and allow access to atouchscreen interface of the platform 600, sensor drivers to obtainsensor readings of sensor circuitry 621 and control and allow access tosensor circuitry 621, EMC drivers to obtain actuator positions of theEMCs 622 and/or control and allow access to the EMCs 622, a cameradriver to control and allow access to an embedded image capture device,audio drivers to control and allow access to one or more audio devices.

The power management integrated circuitry (PMIC) 625 (also referred toas “power management circuitry 625”) may manage power provided tovarious components of the platform 600. In particular, with respect tothe baseband circuitry 610, the PMIC 625 may control power-sourceselection, voltage scaling, battery charging, or DC-to-DC conversion.The PMIC 625 may often be included when the platform 600 is capable ofbeing powered by a battery 630, for example, when the device is includedin a UE 501.

In some embodiments, the PMIC 625 may control, or otherwise be part of,various power saving mechanisms of the platform 600. For example, if theplatform 600 is in an RRC_Connected state, where it is still connectedto the RAN node as it expects to receive traffic shortly, then it mayenter a state known as Discontinuous Reception Mode (DRX) after a periodof inactivity. During this state, the platform 600 may power down forbrief intervals of time and thus save power. If there is no data trafficactivity for an extended period of time, then the platform 600 maytransition off to an RRC_Idle state, where it disconnects from thenetwork and does not perform operations such as channel qualityfeedback, handover, etc. The platform 600 goes into a very LP state andit performs paging where again it periodically wakes up to listen to thenetwork and then powers down again. The platform 600 may not receivedata in this state; in order to receive data, it must transition back toRRC_Connected state. An additional power saving mode may allow a deviceto be unavailable to the network for periods longer than a paginginterval (ranging from seconds to a few hours). During this time, thedevice is totally unreachable to the network and may power downcompletely. Any data sent during this time incurs a large delay and itis assumed the delay is acceptable.

A battery 630 may power the platform 600, although in some examples theplatform 600 may be mounted deployed in a fixed location, and may have apower supply coupled to an electrical grid. The battery 630 may be alithium ion battery, a metal-air battery, such as a zinc-air battery, analuminum-air battery, a lithium-air battery, and the like. In someimplementations, such as in V2X applications, the battery 630 may be atypical lead-acid automotive battery.

In some implementations, the battery 630 may be a “smart battery,” whichincludes or is coupled with a Battery Management System (BMS) or batterymonitoring integrated circuitry. The BMS may be included in the platform600 to track the state of charge (SoCh) of the battery 630. The BMS maybe used to monitor other parameters of the battery 630 to providefailure predictions, such as the state of health (SoH) and the state offunction (SoF) of the battery 630. The BMS may communicate theinformation of the battery 630 to the application circuitry 605 or othercomponents of the platform 600. The BMS may also include ananalog-to-digital (ADC) convertor that allows the application circuitry605 to directly monitor the voltage of the battery 630 or the currentflow from the battery 630. The battery parameters may be used todetermine actions that the platform 600 may perform, such astransmission frequency, network operation, sensing frequency, and thelike.

A power block, or other power supply coupled to an electrical grid maybe coupled with the BMS to charge the battery 630. In some examples, thepower block may be replaced with a wireless power receiver to obtain thepower wirelessly, for example, through a loop antenna in the computerplatform 600. In these examples, a wireless battery charging circuit maybe included in the BMS. The specific charging circuits chosen may dependon the size of the battery 630, and thus, the current required. Thecharging may be performed using the Airfuel standard promulgated by theAirfuel Alliance, the Qi wireless charging standard promulgated by theWireless Power Consortium, or the Rezence charging standard promulgatedby the Alliance for Wireless Power, among others.

User interface circuitry 650 includes various input/output (I/O) devicespresent within, or connected to, the platform 600, and includes one ormore user interfaces designed to enable user interaction with theplatform 600 and/or peripheral component interfaces designed to enableperipheral component interaction with the platform 600. The userinterface circuitry 650 includes input device circuitry and outputdevice circuitry. Input device circuitry includes any physical orvirtual means for accepting an input including, inter alia, one or morephysical or virtual buttons (e.g., a reset button), a physical keyboard,keypad, mouse, touchpad, touchscreen, microphones, scanner, headset,and/or the like. The output device circuitry includes any physical orvirtual means for showing information or otherwise conveyinginformation, such as sensor readings, actuator position(s), or otherlike information. Output device circuitry may include any number and/orcombinations of audio or visual display, including, inter alia, one ormore simple visual outputs/indicators (e.g., binary status indicators(e.g., light emitting diodes (LEDs)) and multi-character visual outputs,or more complex outputs such as display devices or touchscreens (e.g.,Liquid Chrystal Displays (LCD), LED displays, quantum dot displays,projectors, etc.), with the output of characters, graphics, multimediaobjects, and the like being generated or produced from the operation ofthe platform 600. The output device circuitry may also include speakersor other audio emitting devices, printer(s), and/or the like. In someembodiments, the sensor circuitry 621 may be used as the input devicecircuitry (e.g., an image capture device, motion capture device, or thelike) and one or more EMCs may be used as the output device circuitry(e.g., an actuator to provide haptic feedback or the like). In anotherexample, NFC circuitry comprising an NFC controller coupled with anantenna element and a processing device may be included to readelectronic tags and/or connect with another NFC-enabled device.Peripheral component interfaces may include, but are not limited to, anon-volatile memory port, a USB port, an audio jack, a power supplyinterface, etc.

Although not shown, the components of platform 600 may communicate withone another using a suitable bus or interconnect (IX) technology, whichmay include any number of technologies, including ISA, EISA, PCI, PCIx,PCIe, a Time-Trigger Protocol (TTP) system, a FlexRay system, or anynumber of other technologies. The bus/IX may be a proprietary bus/IX,for example, used in a SoC based system. Other bus/IX systems may beincluded, such as an I2C interface, an SPI interface, point-to-pointinterfaces, and a power bus, among others.

FIG. 7 illustrates example components of baseband circuitry 710 and RFEM715 in accordance with various embodiments. The baseband circuitry 710corresponds to the baseband circuitry 610 of FIG. 6 . The RFEM 715corresponds to the RFEM and 615 of FIG. 6 . As shown, the RFEMs 715 mayinclude RF circuitry 706, front-end module (FEM) circuitry 708, antennaarray 711 coupled together at least as shown.

The baseband circuitry 710 includes circuitry and/or control logicconfigured to carry out various radio/network protocol and radio controlfunctions that enable communication with one or more radio networks viathe RF circuitry 706. The radio control functions may include, but arenot limited to, signal modulation/demodulation, encoding/decoding, RFshifting, etc. In some embodiments, modulation/demodulation circuitry ofthe baseband circuitry 710 may include Fast-Fourier Transform (FFT),precoding, or constellation mapping/demapping functionality. In someembodiments, encoding/decoding circuitry of the baseband circuitry 710may include convolution, tail-biting convolution, turbo, Viterbi, or LowDensity Parity Check (LDPC) encoder/decoder functionality. Embodimentsof modulation/demodulation and encoder/decoder functionality are notlimited to these examples and may include other suitable functionalityin other embodiments. The baseband circuitry 710 is configured toprocess baseband signals received from a receive signal path of the RFcircuitry 706 and to generate baseband signals for a transmit signalpath of the RF circuitry 706. The baseband circuitry 710 is configuredto interface with application circuitry 605 (see FIG. 6 ) for generationand processing of the baseband signals and for controlling operations ofthe RF circuitry 706. The baseband circuitry 710 may handle variousradio control functions.

The aforementioned circuitry and/or control logic of the basebandcircuitry 710 may include one or more single or multi-core processors.For example, the one or more processors may include a 3G basebandprocessor 704A, a 4G/LTE baseband processor 704B, a 5G/NR basebandprocessor 704C, or some other baseband processor(s) 704D for otherexisting generations, generations in development or to be developed inthe future (e.g., sixth generation (6G), etc.). In other embodiments,some or all of the functionality of baseband processors 704A-D may beincluded in modules stored in the memory 704G and executed via a CentralProcessing Unit (CPU) 704E. In other embodiments, some or all of thefunctionality of baseband processors 704A-D may be provided as hardwareaccelerators (e.g., FPGAs, ASICs, etc.) loaded with the appropriate bitstreams or logic blocks stored in respective memory cells. In variousembodiments, the memory 704G may store program code of a real-time OS(RTOS), which when executed by the CPU 704E (or other basebandprocessor), is to cause the CPU 704E (or other baseband processor) tomanage resources of the baseband circuitry 710, schedule tasks, etc.Examples of the RTOS may include Operating System Embedded (OSE)™provided by Enea®, Nucleus RTOS™ provided by Mentor Graphics®, VersatileReal-Time Executive (VRTX) provided by Mentor Graphics®, ThreadX™provided by Express Logic®, FreeRTOS, REX OS provided by Qualcomm®, OKL4provided by Open Kernel (OK) Labs®, or any other suitable RTOS, such asthose discussed herein. In addition, the baseband circuitry 710 includesone or more audio digital signal processor(s) (DSP) 704F. The audioDSP(s) 704F include elements for compression/decompression and echocancellation and may include other suitable processing elements in otherembodiments.

In some embodiments, each of the processors 704A-704E include respectivememory interfaces to send/receive data to/from the memory 704G. Thebaseband circuitry 710 may further include one or more interfaces tocommunicatively couple to other circuitries/devices, such as aninterface to send/receive data to/from memory external to the basebandcircuitry 710; an application circuitry interface to send/receive datato/from the application circuitry 605); an RF circuitry interface tosend/receive data to/from RF circuitry 706 of FIG. 7 ; a wirelesshardware connectivity interface to send/receive data to/from one or morewireless hardware elements (e.g., NFC components, Bluetooth®/Bluetooth®Low Energy components, Wi-Fi® components, and/or the like); and a powermanagement interface to send/receive power or control signals to/fromthe PMIC 625.

In alternate embodiments (which may be combined with the above-describedembodiments), baseband circuitry 710 comprises one or more digitalbaseband systems, which are coupled with one another via an interconnectsubsystem and to a CPU subsystem, an audio subsystem, and an interfacesubsystem. The digital baseband subsystems may also be coupled to adigital baseband interface and a mixed-signal baseband subsystem viaanother interconnect subsystem. Each of the interconnect subsystems mayinclude a bus system, point-to-point connections, network-on-chip (NOC)structures, and/or some other suitable bus or interconnect technology,such as those discussed herein. The audio subsystem may include DSPcircuitry, buffer memory, program memory, speech processing acceleratorcircuitry, data converter circuitry such as analog-to-digital anddigital-to-analog converter circuitry, analog circuitry including one ormore of amplifiers and filters, and/or other like components. In anaspect of the present disclosure, baseband circuitry 710 may includeprotocol processing circuitry with one or more instances of controlcircuitry (not shown) to provide control functions for the digitalbaseband circuitry and/or RF circuitry (e.g., the RFEM 715).

Although not shown by FIG. 7 , in some embodiments, the basebandcircuitry 710 includes individual processing device(s) to operate one ormore wireless communication protocols (e.g., a “multi-protocol basebandprocessor” or “protocol processing circuitry”) and individual processingdevice(s) to implement PHY layer functions. In these embodiments, thePHY layer functions include the aforementioned radio control functions.In these embodiments, the protocol processing circuitry operates orimplements various protocol layers/entities of one or more wirelesscommunication protocols. In a first example, the protocol processingcircuitry may operate LTE protocol entities and/or 5G/NR protocolentities when the baseband circuitry 710 and/or RF circuitry 706 arepart of mmWave communication circuitry or some other suitable cellularcommunication circuitry. In the first example, the protocol processingcircuitry would operate MAC, RLC, PDCP, SDAP, RRC, and NAS functions. Ina second example, the protocol processing circuitry may operate one ormore IEEE-based protocols when the baseband circuitry 710 and/or RFcircuitry 706 are part of a Wi-Fi communication system. In the secondexample, the protocol processing circuitry would operate Wi-Fi MAC andlogical link control (LLC) functions. The protocol processing circuitrymay include one or more memory structures (e.g., 704G) to store programcode and data for operating the protocol functions, as well as one ormore processing cores to execute the program code and perform variousoperations using the data. The baseband circuitry 710 may also supportradio communications for more than one wireless protocol.

The various hardware elements of the baseband circuitry 710 discussedherein may be implemented, for example, as a solder-down substrateincluding one or more integrated circuits (ICs), a single packaged ICsoldered to a main circuit board or a multi-chip module containing twoor more ICs. In one example, the components of the baseband circuitry710 may be suitably combined in a single chip or chipset, or disposed ona same circuit board. In another example, some or all of the constituentcomponents of the baseband circuitry 710 and RF circuitry 706 may beimplemented together such as, for example, a SoC or System-in-Package(SiP). In another example, some or all of the constituent components ofthe baseband circuitry 710 may be implemented as a separate SoC that iscommunicatively coupled with and RF circuitry 706 (or multiple instancesof RF circuitry 706). In yet another example, some or all of theconstituent components of the baseband circuitry 710 and the applicationcircuitry 605 may be implemented together as individual SoCs mounted toa same circuit board (e.g., a “multi-chip package”).

In some embodiments, the baseband circuitry 710 may provide forcommunication compatible with one or more radio technologies. Forexample, in some embodiments, the baseband circuitry 710 may supportcommunication with an E-UTRAN or other WMAN, a WLAN, a WPAN. Embodimentsin which the baseband circuitry 710 is configured to support radiocommunications of more than one wireless protocol may be referred to asmulti-mode baseband circuitry.

RF circuitry 706 may enable communication with wireless networks usingmodulated electromagnetic radiation through a non-solid medium. Invarious embodiments, the RF circuitry 706 may include switches, filters,amplifiers, etc. to facilitate the communication with the wirelessnetwork. RF circuitry 706 may include a receive signal path, which mayinclude circuitry to down-convert RF signals received from the FEMcircuitry 708 and provide baseband signals to the baseband circuitry710. RF circuitry 706 may also include a transmit signal path, which mayinclude circuitry to up-convert baseband signals provided by thebaseband circuitry 710 and provide RF output signals to the FEMcircuitry 708 for transmission.

In some embodiments, the receive signal path of the RF circuitry 706 mayinclude mixer circuitry 706 a, amplifier circuitry 706 b and filtercircuitry 706 c. In some embodiments, the transmit signal path of the RFcircuitry 706 may include filter circuitry 706 c and mixer circuitry 706a. RF circuitry 706 may also include synthesizer circuitry 706 d forsynthesizing a frequency for use by the mixer circuitry 706 a of thereceive signal path and the transmit signal path. In some embodiments,the mixer circuitry 706 a of the receive signal path may be configuredto down-convert RF signals received from the FEM circuitry 708 based onthe synthesized frequency provided by synthesizer circuitry 706 d. Theamplifier circuitry 706 b may be configured to amplify thedown-converted signals and the filter circuitry 706 c may be a low-passfilter (LPF) or band-pass filter (BPF) configured to remove unwantedsignals from the down-converted signals to generate output basebandsignals. Output baseband signals may be provided to the basebandcircuitry 710 for further processing. In some embodiments, the outputbaseband signals may be zero-frequency baseband signals, although thisis not a requirement. In some embodiments, mixer circuitry 706 a of thereceive signal path may comprise passive mixers, although the scope ofthe embodiments is not limited in this respect.

In some embodiments, the mixer circuitry 706 a of the transmit signalpath may be configured to up-convert input baseband signals based on thesynthesized frequency provided by the synthesizer circuitry 706 d togenerate RF output signals for the FEM circuitry 708. The basebandsignals may be provided by the baseband circuitry 710 and may befiltered by filter circuitry 706 c.

In some embodiments, the mixer circuitry 706 a of the receive signalpath and the mixer circuitry 706 a of the transmit signal path mayinclude two or more mixers and may be arranged for quadraturedownconversion and upconversion, respectively. In some embodiments, themixer circuitry 706 a of the receive signal path and the mixer circuitry706 a of the transmit signal path may include two or more mixers and maybe arranged for image rejection (e.g., Hartley image rejection). In someembodiments, the mixer circuitry 706 a of the receive signal path andthe mixer circuitry 706 a of the transmit signal path may be arrangedfor direct downconversion and direct upconversion, respectively. In someembodiments, the mixer circuitry 706 a of the receive signal path andthe mixer circuitry 706 a of the transmit signal path may be configuredfor super-heterodyne operation.

In some embodiments, the output baseband signals and the input basebandsignals may be analog baseband signals, although the scope of theembodiments is not limited in this respect. In some alternateembodiments, the output baseband signals and the input baseband signalsmay be digital baseband signals. In these alternate embodiments, the RFcircuitry 706 may include analog-to-digital converter (ADC) anddigital-to-analog converter (DAC) circuitry and the baseband circuitry710 may include a digital baseband interface to communicate with the RFcircuitry 706.

In some dual-mode embodiments, a separate radio IC circuitry may beprovided for processing signals for each spectrum, although the scope ofthe embodiments is not limited in this respect.

In some embodiments, the synthesizer circuitry 706 d may be afractional-N synthesizer or a fractional N/N+1 synthesizer, although thescope of the embodiments is not limited in this respect as other typesof frequency synthesizers may be suitable. For example, synthesizercircuitry 706 d may be a delta-sigma synthesizer, a frequencymultiplier, or a synthesizer comprising a phase-locked loop with afrequency divider.

The synthesizer circuitry 706 d may be configured to synthesize anoutput frequency for use by the mixer circuitry 706 a of the RFcircuitry 706 based on a frequency input and a divider control input. Insome embodiments, the synthesizer circuitry 706 d may be a fractionalN/N+1 synthesizer.

In some embodiments, frequency input may be provided by a voltagecontrolled oscillator (VCO), although that is not a requirement. Dividercontrol input may be provided by either the baseband circuitry 710 orthe application circuitry 605 depending on the desired output frequency.In some embodiments, a divider control input (e.g., N) may be determinedfrom a look-up table based on a channel indicated by the applicationcircuitry 605.

Synthesizer circuitry 706 d of the RF circuitry 706 may include adivider, a delay-locked loop (DLL), a multiplexer and a phaseaccumulator. In some embodiments, the divider may be a dual modulusdivider (DMD) and the phase accumulator may be a digital phaseaccumulator (DPA). In some embodiments, the DMD may be configured todivide the input signal by either N or N+1 (e.g., based on a carry out)to provide a fractional division ratio. In some example embodiments, theDLL may include a set of cascaded, tunable, delay elements, a phasedetector, a charge pump and a D-type flip-flop. In these embodiments,the delay elements may be configured to break a VCO period up into Ndequal packets of phase, where Nd is the number of delay elements in thedelay line. In this way, the DLL provides negative feedback to helpensure that the total delay through the delay line is one VCO cycle.

In some embodiments, synthesizer circuitry 706 d may be configured togenerate a carrier frequency as the output frequency, while in otherembodiments, the output frequency may be a multiple of the carrierfrequency (e.g., twice the carrier frequency, four times the carrierfrequency) and used in conjunction with quadrature generator and dividercircuitry to generate multiple signals at the carrier frequency withmultiple different phases with respect to each other. In someembodiments, the output frequency may be a LO frequency (fLO). In someembodiments, the RF circuitry 706 may include an IQ/polar converter.

FEM circuitry 708 may include a receive signal path, which may includecircuitry configured to operate on RF signals received from antennaarray 711, amplify the received signals and provide the amplifiedversions of the received signals to the RF circuitry 706 for furtherprocessing. FEM circuitry 708 may also include a transmit signal path,which may include circuitry configured to amplify signals fortransmission provided by the RF circuitry 706 for transmission by one ormore of antenna elements of antenna array 711. In various embodiments,the amplification through the transmit or receive signal paths may bedone solely in the RF circuitry 706, solely in the FEM circuitry 708, orin both the RF circuitry 706 and the FEM circuitry 708.

In some embodiments, the FEM circuitry 708 may include a TX/RX switch toswitch between transmit mode and receive mode operation. The FEMcircuitry 708 may include a receive signal path and a transmit signalpath. The receive signal path of the FEM circuitry 708 may include anLNA to amplify received RF signals and provide the amplified received RFsignals as an output (e.g., to the RF circuitry 706). The transmitsignal path of the FEM circuitry 708 may include a power amplifier (PA)to amplify input RF signals (e.g., provided by RF circuitry 706), andone or more filters to generate RF signals for subsequent transmissionby one or more antenna elements of the antenna array 711.

The antenna array 711 comprises one or more antenna elements, each ofwhich is configured convert electrical signals into radio waves totravel through the air and to convert received radio waves intoelectrical signals. For example, digital baseband signals provided bythe baseband circuitry 710 is converted into analog RF signals (e.g.,modulated waveform) that will be amplified and transmitted via theantenna elements of the antenna array 711 including one or more antennaelements (not shown). The antenna elements may be omnidirectional,direction, or a combination thereof. The antenna elements may be formedin a multitude of arranges as are known and/or discussed herein. Theantenna array 711 may comprise microstrip antennas or printed antennasthat are fabricated on the surface of one or more printed circuitboards. The antenna array 711 may be formed in as a patch of metal foil(e.g., a patch antenna) in a variety of shapes, and may be coupled withthe RF circuitry 706 and/or FEM circuitry 708 using metal transmissionlines or the like.

Processors of the application circuitry 605 and processors of thebaseband circuitry 710 may be used to execute elements of one or moreinstances of a protocol stack. For example, processors of the basebandcircuitry 710, alone or in combination, may be used execute Layer 3,Layer 2, or Layer 1 functionality, while processors of the applicationcircuitry 605 may utilize data (e.g., packet data) received from theselayers and further execute Layer 4 functionality (e.g., TCP and UDPlayers). As referred to herein, Layer 3 may comprise a RRC layer. Asreferred to herein, Layer 2 may comprise a MAC layer, an RLC layer, anda PDCP layer. As referred to herein, Layer 1 may comprise a PHY layer ofa UE/RAN node.

EXAMPLES

Example 1 may include a new SDP attribute to indicate capabilities forcarriage of 360 video as part of an RTP stream including one or more ofthe following: ability to carry immersive media metadata information aspart of the RTP payload format, e.g., using supplemental informationenhancement (SEI) messages, e.g., information carried in theomnidirectional video specific SEI messages are then to be used in therendering phase at the receiver.

Example 2 may include a new SDP attribute to indicate viewport-dependentprocessing capabilities for carriage of 360 video as part of an RTPstream including one or more of the following: ability to signal desiredviewport using an RTCP feedback message; and/or ability to signal theactually transmitted viewport using an RTP header extension message.

Example 3 may include SDP attribute of example 2 or some other exampleherein, where as a result two or more RTP streams may be negotiated.

Example 4 may include the single SDP attribute that combines thecapabilities in examples 1 and 2 or some other example herein.

Example 5 may include SDP attribute of example 3 or some other exampleherein, where one RTP stream is for the base 360 video and another is aviewport-optimized RTP stream, with the high quality 360 videocorresponding to the desired viewport.

Example 6 may include SDP attribute of example 1 or some other exampleherein, where the omnidirectional video specific SEI messages maycontain one or more of the following: the equirectangular projection SEImessage, the cubemap projection SEI message, the sphere rotation SEImessage, and the region-wise packing SEI message.

Example 7 may include a method comprising: receiving an SDP offermessage, the SDP offer message to indicate capabilities for support of360 video in an RTP stream; encoding, for transmission, an SDP answermessage based on the SDP offer message, the SDP answer message toindicate a confirmation of the capabilities.

Example 8 may include the method of example 7 or another example herein,wherein the SDP offer message further includes a viewport-dependentprocessing capability.

Example 9 may include the method of example 8 or another example herein,wherein the SDP offer message includes two or more RTP streams when theSDP answer message includes the viewport-dependent processingcapability.

Example 10 may include the method of example 8-9 or another exampleherein, wherein the capabilities for support of 360 video and theviewport-dependent processing capability are indicated in a same SDPattribute in the SDP offer message.

Example 11 may include the method of example 8-10 or another exampleherein, wherein the SDP answer includes an acceptance of theviewport-dependent processing capability.

Example 12 may include the method of example 8-11 or another exampleherein, further comprising encoding, for transmission, immersive mediaincluding 360 video based on the SDP offer message.

Example 13 may include the method of example 8-12 or another exampleherein, wherein the SDP offer message is received from a UE.

Example 14 may include the method of example 8-13 or another exampleherein, wherein the method is performed by a conferencing server or aportion thereof.

Example 15 may include a method comprising: encoding, for transmission,an SDP offer message, the SDP offer message to indicate capabilities forsupport of 360 video in an RTP stream; receiving an SDP answer messagebased on the SDP offer message, the SDP answer message to indicate aconfirmation of the capabilities.

Example 16 may include the method of example 15 or another exampleherein, wherein the SDP offer message further includes aviewport-dependent processing capability.

Example 17 may include the method of example 16 or another exampleherein, wherein the SDP offer message includes two or more RTP streamswhen the SDP answer message includes the viewport-dependent processingcapability.

Example 18 may include the method of example 16-17 or another exampleherein, wherein the capabilities for support of 360 video and theviewport-dependent processing capability are indicated in a same SDPattribute in the SDP offer message.

Example 19 may include the method of example 16-18 or another exampleherein, wherein the SDP answer includes an acceptance of theviewport-dependent processing capability.

Example 20 may include the method of example 16-19 or another exampleherein, further comprising receiving immersive media including 360 videobased on the SDP offer message.

Example 21 may include the method of example 16-20 or another exampleherein, wherein the SDP answer message is received from a conferencingserver.

Example 22 may include the method of example 16-21 or another exampleherein, wherein the method is performed by a UE or a portion thereof.

Example 23 includes an electronic device that comprises: first circuitryto decode, based on a first real-time transport protocol (RTP) stream,first visual data related to a plurality of images concurrently taken ofa location; second circuitry to decode, based on a session descriptionprotocol (SDP) offer received from a user equipment (UE), an indicationthat the UE supports immersive viewing capability; and third circuitryto transmit, via a second RTP stream based on the SDP offer, secondvisual data related to the first visual data, wherein the second visualdata includes a supplemental information enhancement (SEI) message to beused to display at least a portion of the second visual data.

Example 24 include the electronic device of example 23, wherein thefirst RTP stream includes visual data related to two or more of theplurality of images.

Example 25 include the electronic device of example 23, wherein thefirst RTP stream include visual data related to an image of theplurality of images, and wherein the electronic device furthercomprises: fourth circuitry to decode, based on a third RTP stream,third visual data related to a another image of the plurality of images;and fifth circuitry to stitch together the first visual data and thethird visual data.

Example 26 include the electronic device of any of examples 23-25,wherein the SEI message is an equirectangular projection SEI message, acubemap projection SEI message, a sphere rotation SEI message, or aregion-wise packing SEI message.

Example 27 include the electronic device of any of examples 23-25,further comprising sixth circuitry to decode, based on a RTP controlprotocol (RTCP) feedback message received from the UE, an indication ofa desired viewing orientation of the first visual data.

Example 28 include the electronic device of example 27, wherein thesecond visual data is a portion of the first visual data that is basedon the indication of the desired viewing orientation.

Example 29 include the electronic device of any of examples 23-25,wherein the first, second, and third circuitry are circuitry of aprocessor.

Example 30 includes an electronic device comprising: first circuitry todetermine, based on a received real-time transport protocol (RTP) streamthat includes visual data related to a plurality of images concurrentlytaken of a location, an elementary stream; second circuitry to decode,based on the elementary stream, the visual data and a supplementalinformation enhancement (SEI) message; third circuitry to generate,based on the visual data and the SEI message, a mapping of the visualdata to a visual field; and fourth circuitry to output, to a displaydevice, data related to the mapping of the visual data to the visualfield.

Example 31 include the electronic device of example 30, wherein theelectronic device is a user equipment (UE) of a third generationpartnership project (3GPP) network, and wherein the UE includes thedisplay device.

Example 32 include the electronic device of example 30, wherein the SEImessage is an equirectangular projection SEI message, a cubemapprojection SEI message, a sphere rotation SEI message, or a region-wisepacking SEI message.

Example 33 include the electronic device of any of examples 30-32,further comprising fifth circuitry to facilitate transmission, in a RTPcontrol protocol (RTCP) feedback message, an indication of a desiredviewing orientation of the first visual data.

Example 34 include the electronic device of any of examples 30-32,wherein the RTP stream is a first RTP stream that includes first visualdata related to a first image of the plurality of images, and whereinthe elementary stream is further based on a decoded second RTP streamthat includes second visual data related to a second image of theplurality of images.

Example 35 include the electronic device of any of examples 30-32,wherein the RTP stream includes visual data related to at least a firstimage and a second image of the plurality of images.

Example 36 include the electronic device of any of examples 30-32,wherein the visual data is a high efficiency video coding (HEVC)bitstream that includes the SEI message.

Example 37 includes a method comprising: transmitting, from a userequipment (UE) to an (IP) multimedia subsystem (IMS) server, a sessiondescription protocol (SDP) offer that includes an indication that the UEsupports immersive video; determining, by the UE based on a SDP responsereceived from the server, an indication that server supports immersivevideo; determining, based on the SDP offer and the SDP response, areal-time transport protocol (RTP) media flow that includes visual datarelated to a plurality of images concurrently taken of a location and asupplemental information enhancement (SEI) message that is to be used todisplay at least a portion of the visual data; and visually displaying,by the UE based on the visual data and the SEI message, the portion ofthe visual data to a user of the user device.

Example 38 include the method of example 37, wherein the SDP offerincludes an indication that the UE supports viewport-dependentprocessing.

Example 39 include the method of example 37, wherein the SDP responseincludes an indication that the server supports viewport-dependentprocessing.

Example 40 include the method of any of examples 37-39, furthercomprising transmitting, from the UE to the server in a RTP controlprotocol (RTCP) feedback message, an indication of a desired field ofview of the visual data.

Example 41 include the method of example 40, wherein the visual data ofthe RTP media flow is based on the indication of the desired field ofview.

Example 42 include the method of any of examples 37-39, wherein thevisual data is based on an Omnidirectional Media Format (OMAF) videoprofile.

Example 43 may include an apparatus comprising means to perform one ormore elements of a method described in or related to any of examples1-42, or any other method or process described herein.

Example 44 may include one or more non-transitory computer-readablemedia comprising instructions to cause an electronic device, uponexecution of the instructions by one or more processors of theelectronic device, to perform one or more elements of a method describedin or related to any of examples 1-42, or any other method or processdescribed herein.

Example 45 may include an apparatus comprising logic, modules, orcircuitry to perform one or more elements of a method described in orrelated to any of examples 1-42, or any other method or processdescribed herein.

Example 46 may include a method, technique, or process as described inor related to any of examples 1-42, or portions or parts thereof.

Example 47 may include an apparatus comprising: one or more processorsand one or more computer-readable media comprising instructions that,when executed by the one or more processors, cause the one or moreprocessors to perform the method, techniques, or process as described inor related to any of examples 1-42, or portions thereof.

Example 48 may include a signal as described in or related to any ofexamples 1-42, or portions or parts thereof.

Example 49 may include a datagram, packet, frame, segment, protocol dataunit (PDU), or message as described in or related to any of examples1-42, or portions or parts thereof, or otherwise described in thepresent disclosure.

Example 50 may include a signal encoded with data as described in orrelated to any of examples 1-42, or portions or parts thereof, orotherwise described in the present disclosure.

Example 51 may include a signal encoded with a datagram, packet, frame,segment, PDU, or message as described in or related to any of examples1-42, or portions or parts thereof, or otherwise described in thepresent disclosure.

Example 52 may include an electromagnetic signal carryingcomputer-readable instructions, wherein execution of thecomputer-readable instructions by one or more processors is to cause theone or more processors to perform the method, techniques, or process asdescribed in or related to any of examples 1-42, or portions thereof.

Example 53 may include a computer program comprising instructions,wherein execution of the program by a processing element is to cause theprocessing element to carry out the method, techniques, or process asdescribed in or related to any of examples 1-42, or portions thereof.

Example 54 may include a signal in a wireless network as shown anddescribed herein.

Example 55 may include a method of communicating in a wireless networkas shown and described herein.

Example 56 may include a system for providing wireless communication asshown and described herein.

Example 57 may include a device for providing wireless communication asshown and described herein.

Any of the above-described examples may be combined with any otherexample (or combination of examples), unless explicitly statedotherwise. The foregoing description of one or more implementationsprovides illustration and description, but is not intended to beexhaustive or to limit the scope of embodiments to the precise formdisclosed. Modifications and variations are possible in light of theabove teachings or may be acquired from practice of various embodiments.

1. An apparatus comprising: first circuitry to determine, based on areceived real-time transport protocol (RTP) stream that includes visualdata related to a plurality of images concurrently taken of a location,an elementary stream; second circuitry to decode, based on theelementary stream, the visual data and a supplemental enhancementinformation (SEI) message; third circuitry to generate, based on thevisual data and the SEI message, a mapping of the visual data to avisual field; and fourth circuitry to output, to a display device, datarelated to the mapping of the visual data to the visual field.
 2. Theapparatus of claim 1, wherein the apparatus comprises a user equipment(UE) of a third generation partnership project (3GPP) network, andwherein the UE includes the display device.
 3. The apparatus of claim 1,wherein the SEI message is an equirectangular projection SEI message, acubemap projection SEI message, a sphere rotation SEI message, or aregion-wise packing SEI message.
 4. The apparatus of claim 1, furthercomprising fifth circuitry to facilitate transmission, in a RTP controlprotocol (RTCP) feedback message, an indication of a desired viewingorientation of the first visual data.
 5. The apparatus of claim 4,wherein the first visual data is based on the indication of the desiredfield of view.
 6. The apparatus of claim 1, wherein the RTP stream is afirst RTP stream that includes first visual data related to a firstimage of the plurality of images, and wherein the elementary stream isfurther based on a decoded second RTP stream that includes second visualdata related to a second image of the plurality of images.
 7. Theapparatus of claim 1, wherein the RTP stream includes visual datarelated to at least a first image and a second image of the plurality ofimages.
 8. The apparatus of claim 1, wherein the visual data is a highefficiency video coding (HEVC) bitstream that includes the SEI message.9. The apparatus of claim 1, wherein the first circuitry determines,based on the received real-time transport protocol (RTP) stream i) for avideo conference between two or more devices that include the apparatusand ii) that includes the visual data related to the plurality of imagesconcurrently taken of the location, the elementary stream.
 10. Theapparatus of claim 1, wherein the visual data is based on anOmnidirectional Media Format (OMAF) video profile.
 11. A methodcomprising: obtaining, based on a received real-time transport protocol(RTP) stream that includes visual data related to a plurality of imagesa for a video conference between two or more devices, the visual dataand a supplemental enhancement information (SEI) message; generating,based on the visual data and the SEI message, a mapping of the visualdata to a visual field; and outputting, to a display device, datarelated to the mapping of the visual data to the visual field.
 12. Themethod of claim 11, wherein obtaining the visual data and the SEImessage comprises obtaining, by one or more baseband processors, thevisual data and the SEI message.
 13. The method of claim 11, wherein theSEI message is an equirectangular projection SEI message, a cubemapprojection SEI message, a sphere rotation SEI message, or a region-wisepacking SEI message.
 14. The method of claim 11, comprisingtransmitting, in a RTP control protocol (RTCP) feedback message, anindication of a desired viewing orientation of the first visual data.15. The method of claim 14, wherein the first visual data is based onthe indication of the desired field of view.
 16. The method of claim 11,wherein: the RTP stream is a first RTP stream that includes first visualdata related to a first image of the plurality of images; and obtainingthe visual data and the SEI message comprises obtaining the visual dataand the SEI message based on the first RTP stream and a second RTPstream that includes second visual data related to a second image of theplurality of images.
 17. The method of claim 11, wherein the RTP streamincludes visual data related to at least a first image and a secondimage of the plurality of images.
 18. The method of claim 11, whereinthe visual data is a high efficiency video coding (HEVC) bitstream thatincludes the SEI message.
 19. The method of claim 11, wherein theplurality of images were substantially concurrently taken of a location.20. The method of claim 11, wherein the visual data is based on anOmnidirectional Media Format (OMAF) video profile.